Developing  a  Framework  for  Control  of 
Agile  Aircraft  Platforms  in  Autonomous  Hover 


Kyle  J.  Krogh 


A  thesis  submitted  in  partial  fulfillment 
of  the  requirements  for  the  degree  of 


Master  of  Science  in  Aeronautics  &  Astronautics 


University  of  Washington 


2009 


Program  Authorized  to  Offer  Degree:  Aeronautics  &  Astronautics 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

01  MAR  2009  N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Developing  a  Framework  for  Control  of  Agile  Aircraft  Platforms  in 
Autonomous  Hover 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

University  of  Washington 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

CI09-0045 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

18.  NUMBER  19a.  NAME  OF 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  |J|J 

unclassified  unclassified  unclassified 

74 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


University  of  Washington 
Graduate  School 


This  is  to  certify  that  1  have  examined  this  copy  of  a  master’s  thesis  by 


Kyle  J.  Krogh 


and  have  found  that  it  is  complete  and  satisfactory  in  all  respects, 
and  that  any  and  all  revisions  required  by  the  final 
examining  committee  have  been  made. 


Committee  Members: 


Kristi  A.  Morgansen 


Juris  Vagners 


Date: 


In  presenting  this  thesis  in  partial  fulfillment  of  the  requirements  for  a  master’s 
degree  at  the  University  of  Washington,  I  agree  that  the  Library  shall  make  its  copies 
freely  available  for  inspection.  I  further  agree  that  extensive  copying  of  this  thesis  is 
allowable  only  for  scholarly  purposes,  consistent  with  “fair  use”  as  prescribed  in  the 
U.S.  Copyright  Law.  Any  other  reproduction  for  any  purpose  or  by  any  means  shall 
not  be  allowed  without  my  written  permission. 


S  ignat  ure _ 


Date _ 


University  of  Washington 


Abstract 

Developing  a  Framework  for  Control  of 
Agile  Aircraft  Platforms  in  Autonomous  Hover 

Kyle  J.  Krogh 

Chair  of  the  Supervisory  Committee: 

Professor  Kristi  A.  Morgansen 
Aeronautics  &  Astronautics 

This  thesis  extends  previous  work  on  autonomous  agile  aircraft  to  develop  a  frame¬ 
work  for  control  in  both  linear  and  non-linear  flight  profiles.  Two  dynamical  systems 
are  considered,  a  scale  YAK-54  aerobatic  remote  control  aircraft  and  the  Flexrotor 
concept  developed  by  Aerovel.  Both  models  are  capable  of  hover  and  level  flight, 
requiring  complex  control  transitions  between  the  flight  regimes.  Linear  controllers 
are  developed  for  both  models  in  a  hover  configuration.  Open-loop  control  sequences 
for  the  transitions  from  hover  to  level  flight  and  back  are  discussed.  All  controllers  are 
optimized,  and  robustness  considered;  additionally,  a  procedure  for  providing  near- 
optimal  non-linear  control  sequences  is  proposed.  Finally,  a  general  framework  for 
generating  control  laws  of  agile  aircraft  systems  is  put  forth.  Simulation  results  are 
shown  to  support  the  theoretical  analysis. 
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Chapter  1 

INTRODUCTION 

Control  theory  is  the  study  of  dynamical  systems  and  generating  inputs.  Types  of 
dynamical  systems  include  thermal,  electrical,  and  mechanical  systems.  Mechanical 
systems  are  of  specific  concern,  as  unmanned  aerial  vehicles  (UAVs)  have  become 
increasingly  prevalent  and  important  in  the  twenty-first  century.  With  the  growth  of 
UAVs,  new  flight  regimes  are  possible  that  require  the  application  of  linear  and  non¬ 
linear  control  theory  for  automatic  input  generation.  The  topic  of  this  thesis  is  the 
design  of  controllers  for  agile  aircraft  systems  and  the  creation  of  a  broad  framework 
applicable  to  this  class  of  problems. 

1 . 1  Motivation 

The  rapid  development  of  unmanned  aerial  vehicles  has  been  made  possible,  in  part, 
by  recent  advances  in  communication,  computation,  and  sensing.  The  demand  for 
unmanned  systems  has  encouraged  growth  as  well,  and  UAVs  have  numerous  mili¬ 
tary  and  civil  applications.  Military  applications  include  reconnaissance,  surveillance, 
intelligence,  communications,  and  tactical  strike  of  ground  targets,  while  civil  appli¬ 
cations  include  research,  weather  forecasting,  border  patrol,  and  others. 

Unmanned  aircraft  also  offer  a  unique  set  of  advantages  when  compared  to  piloted 
aircraft.  Without  a  pilot,  the  aircraft  can  be  made  smaller  and  lighter  weight  by 
eliminating  life  support  systems.  Unmanned  systems  may  also  be  designed  around 
specific  missions  and  maneuvers  that  are  incompatible  with  a  human  pilot  on  board 
the  aircraft,  such  as  extended  hover  in  fixed-wing  aircraft.  By  eliminating  the  pilot, 
systems  can  be  made  smaller,  allowing  them  to  operate  in  tight  spaces,  to  be  man 
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portable,  or  to  enjoy  unique  configurations. 

Recent  interest  has  been  focused  on  UAVs  capable  of  sustained  hover,  allowing  a 
unique  set  of  capabilities.  Sustained  hovering  flight  in  a  vertical  orientation  is  termed 
thrust-borne  flight,  as  the  propulsion  system  counteracts  gravity  to  keep  the  aircraft 
aloft.  Traditional  flight  with  lift  produced  by  the  wings  is  termed  wing-borne  flight. 
Systems  capable  of  both  thrust-borne  flight  and  wing-borne  flight  strike  a  unique 
balance  between  the  endurance  of  fixed-wing  aircraft  and  the  ability  to  operate  in 
confined  spaces  of  rotary-wing  aircraft.  Consider  the  task  of  observation  and  recon- 


Figure  1.1:  A  ScanEagle  UAV  and  the  SkyHook  Retrieval  System  [28]. 


naissance  in  an  urban  environment.  An  agile  UAV  could  employ  a  belly  camera  able 
to  track  a  target  in  wing-borne  and  thrust-borne  flight,  and  in  an  urban  environ¬ 
ment  could  track  while  in  thrust-borne  flight.  Alternatively,  a  UAV  could  employ 
thrust-borne  flight  as  an  object  avoidance  maneuver  while  operating  in  an  urban  en¬ 
vironment,  or  any  other  similarly  constrained  environment.  Lastly,  a  thrust-borne 
capability  allows  for  vertical  take-off  and  landing,  further  extending  the  operating 
envelope  of  agile  UAVs.  The  Insitu/Boeing  ScanEagle  demonstrates  the  potential  for 
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the  concept  of  non-traditional  launch  and  recovery  methods  via  its  SkyHook  Retrieval 
System,  as  shown  in  Figure  1.1. 

1.2  Problem  Statement 

The  goal  of  this  thesis  is  to  address  the  control  of  two  agile  autonomous  systems, 
a  scale  YAK-54  (see  Figure  1.2)  and  the  Aerovel  Flexrotor  (see  Figure  1.3).  Both 
systems  are  capable  of  wing-borne  and  thrust-borne  flight,  though  they  differ  signif¬ 
icantly  in  design.  The  application  of  previous  work,  combined  with  the  analysis  of 
both  systems,  provides  the  basis  for  a  framework  describing  control  methods  of  agile 
autonomous  systems. 


Figure  1.2:  A  YAK-54  in  hover  in  the  Real  Flight  RC  Simulator  [24], 


Existing  theory  applies  to  various  iterations  of  fixed-wing  aircraft  capable  of  au¬ 
tonomous  hover,  such  as  the  YAK-54,  but  the  design  of  the  Flexrotor  presents  a  new 
approach  to  agile  UAVs  and  the  controls  required.  Recent  work  has  been  conducted 
into  the  various  aspects  of  thrust-borne  flight  and  transition  to  wing-borne  flight,  and 
this  thesis  attempts  to  build  upon  those  efforts. 

The  main  challenge  in  these  problems  is  controlling  aircraft  attitude  and  position 
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while  in  thrust-borne  flight.  Initially,  controllers  are  designed  with  the  assumption 
of  full-state,  non-noisy  feedback,  but  the  final  control  laws  for  each  system  should  be 
robust  against  system  and  measurement  noise,  and  environmental  uncertainty,  such 
as  wind  effects. 

The  proposed  framework  for  agile  UAVs  should  be  applicable  to  a  wide  range 
of  agile  aircraft.  Additionally,  control  methods  for  both  linear  and  non-linear  flight 
regimes  should  be  identified.  Linear  controls  should  be  optimal  under  the  guidelines 
of  LQ  theory,  while  non-linear  flight  regimes  should  be  near-optimal  for  the  desired 
maneuver.  Various  machine-learning  schemes  can  be  used  to  ensure  near-optimality. 
All  proposed  control  methods  should  be  robust  to  noise  and  disturbances  as  well. 


1.3  Previous  Work 

The  general  class  of  problems  considered  here  combines  elements  of  linear  control  and 
non-linear  control.  Specific  examples  of  both  control  elements  have  been  previously 
studied.  Existing  work  on  both  linear  control  and  non-linear  control  for  various  agile 
autonomous  systems  will  be  summarized  in  this  section. 


Figure  1.3:  The  Aerovel  Flexrotor  concept. 
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1.3.1  Linear  Control  Methods  for  Hover 

The  flight  regimes  that  can  be  reasonably  approximated  as  linear  in  behavior  comprise 
two  different  categories,  thrust-borne  and  wing-borne  flight.  Aircraft  behavior  while 
in  wing-borne  flight  has  been  extensively  studied  and  is  cataloged  by  Stevens  [25], 
McCormick  [16],  Yechout  [30],  and  others.  The  case  of  thrust-borne  flight  represents 
the  general  class  of  problems  addressed  here  and  is  motivated  by  various  attempts  and 
approaches  to  controlling  an  agile  aircraft  in  a  hover,  or  thrust-borne,  configuration. 

In  the  work  by  Green  [9],  a  micro- air- vehicle  (MAV)  is  considered,  and  simplified 
dynamics  are  proposed  to  model  a  MAV  in  transition  from  level  flight  to  hover.  While 
the  MAV  is  in  hovering  flight,  the  thrust  must  balance  the  aircraft  weight  and  drag 
forces,  which  are  expected  to  be  small.  The  dynamics  fail  to  consider  the  issue  of 
controlling  the  aircraft  while  in  hover,  or  how  to  account  for  rotational  effects  due  to 
propeller  inertia. 

A  novel  tail-sitter  UAV  design  proposed  by  Stone  [27]  used  multi-disciplinary 
optimization  schemes.  Subsequent  work  [26,  6]  developed  a  hover  control  model  using 
Linear  Quadratic  Regulator  (LQR)  techniques  to  stabilize  the  vehicle  in  the  presence 
of  prescribed  wind  gusts.  The  four  low-level  controllers  that  make  up  the  vertical  flight 
controller  are  a  velocity  elevator  controller,  a  velocity  rudder  controller,  a  vertical  roll 
rate  controller,  and  a  vertical  velocity  throttle  controller.  The  controller  is  then 
incorporated  directly  into  the  optimization  problem. 

In  a  related  work  [8],  a  small  foam  agile  autonomous  aircraft  is  considered  that 
relics  on  motion  capture  cameras  for  state  sensing  and  computes  all  controls  off-board. 
The  hover  controller  uses  full  state  feedback  from  the  sensing  system,  and  controller 
gains  are  optimized  using  linear  quadratic  control  techniques.  Large  penalties  are 
assigned  to  pitch,  roll,  and  their  respective  derivatives  to  ensure  that  the  vehicle 
maintains  thrust-borne  flight.  Small  penalties  on  the  position  deviations  prevent  the 
vehicle  from  making  sudden  movements  should  the  position  change.  A  large  penalty 
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is  used  on  altitude  deviation,  ensuring  that  the  airplane  can  accurately  maintain  a 
prescribed  altitude.  As  foam  airplanes  are  low  in  structural  rigidity,  airframe  twist  is 
a  concern  and  is  mitigated  by  focusing  on  minimizing  rapid  changes  in  control  surface 
position. 

The  same  experimental  setup  is  used  in  [21]  to  investigate  a  controls  framework 
based  on  logic  to  operate  in  the  various  flight  regimes  of  an  agile  aircraft.  Using 
the  motion  capture  system,  local  state-feedback  stabilizers  are  proposed  for  set-point 
hovering  flight  and  for  set-point  hovering  flight  at  angle  9.  The  stabilizers  are  designed 
following  optimal  LQR  techniques. 

A  different  approach  was  explored  by  Johnson  [10],  in  which  the  use  of  dynamic 
inversion  with  neural  network  adaptation  was  considered.  The  adaptive  controller 
capable  of  transitioning  to  and  from  hovering  flight  in  a  nearly  stationary  position 
was  applied  to  the  GTEdge  UAV.  The  controller  was  validated  with  eight  successful 
transitions  from  forward  horizontal  flight  to  hover  and  back  in  varying  atmospheric 
conditions,  though  lateral  and  rotational  drifts  occurred  while  in  hover. 

These  results  have  demonstrated  a  wide  approach  to  controlling  an  agile  aircraft 
in  hover  configuration,  though  linear  quadratic  based  approaches  are  favored.  Less 
focus  has  been  paid  to  controlling  the  translation  or  rotation  of  an  aircraft  in  hover, 
or  to  the  use  of  reduced  order  controllers. 

1.3.2  Non-Linear  Control  Methods 

Transferring  an  agile  aircraft  from  one  flight  regime  to  another  and  back  poses  sig¬ 
nificant  difficulties,  as  the  aircraft  transits  highly  non-linear  ranges  of  its  dynamics. 
In  [8],  transitions  were  performed  in  essentially  an  open-loop  manner.  Transitioning 
to  level  flight  was  performed  by  an  instant  movement  of  the  reference  way  point  to  a 
distant  location  combined  with  a  5%  reduction  in  throttle.  The  transition  to  hover 
was  performed  in  a  prescribed  pitch-up  maneuver,  with  the  ailerons  controlled  by  a 
feedback  loop  to  keep  the  aircraft  on  the  predefined  trajectory.  The  transitions  incor- 
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porated  into  the  hybrid  control  scheme  developed  by  Naldi  [21]  are  again  open  loop, 
but  are  based  upon  commanded  human  pilot  inputs  from  a  large  sequence  of  flight 
tests. 

Taking  a  different  approach,  the  transition  controllers  implemented  by  Johnson 

[10]  involved  the  use  of  a  trajectory  generator.  Two  different  transitions  were  con¬ 
sidered  for  each  transition,  one  slower  and  one  faster.  The  slower  transition  involved 
a  gradual  and  linear  ramping  of  the  commanded  pitch  angle  to  90  degrees  combined 
with  a  decrease  in  commanded  velocity  over  a  ten  second  period.  The  faster  scheme 
involved  a  step  change  in  pitch  angle  to  120  degrees  and  then  back  to  90  degrees  while 
the  commanded  velocity  decreased  to  zero  in  only  five  seconds.  The  transitions  back 
to  level  flight  were  essentially  inverses  of  these  two  schemes. 

While  open  loop  controllers  are  popular  for  their  ease  of  use  and  simplicity,  opti¬ 
mizing  the  performance  of  such  controllers  is  much  more  challenging  than  optimizing 
linear  feedback  controllers.  In  the  attempt  to  provide  measures  of  optimality,  machine 
learning  approaches  have  been  developed,  investigated,  and  tested.  Kearns  and  Singh 

[11]  presented  a  reinforcement  learning  (RL)  algorithm  to  find  near-optimal  returns 
in  Markov  decision  processes  (MDP).  They  felt  that  previous  work  lacked  provable 
bounds  on  the  resources  required  to  learn  an  optimal  (or  near-optimal)  policy.  The 
algorithm  they  developed,  known  as  Ei  for  Explicit  Explore  or  Exploit,  provided 
guarantees  on  the  resources  and  time  necessary  to  find  a  near-optimal  policy  through 
a  mix  of  balanced  wandering  and  exploitation  of  areas  within  the  state  space. 

While  E 3  was  a  significant  step  forward,  Abbeel  and  Ng  [4]  felt  that  the  bias  on 
exploration  made  E 3  incompatible  with  a  real  system.  To  focus  more  on  exploitation, 
they  developed  a  three  step  algorithm  referred  to  as  apprenticeship  learning.  First,  a 
teacher  demonstrates  a  policy,  and  all  state-action  trajectories  are  recorded.  Second, 
using  any  RL  algorithm  and  all  state-action  trajectories  in  the  training  set,  a  near- 
optimal  policy  is  found  for  the  system.  Finally,  the  near-optimal  policy  is  compared  to 
the  teacher’s  policy.  If  the  performance  meets  or  exceeds  the  teacher’s,  the  algorithm 


is  finished;  otherwise,  the  trajectories  are  added  to  the  state-action  trajectories  from 
the  unsuccesful  test  to  the  training  step  and  the  algorithm  loops  back  to  the  second 
step. 

This  approach  to  reinforcement  learning  was  subsequently  applied  to  an  autonomous 
aerobatic  helicopter  [2,  3],  which  represented  the  first  successful  autonomous  comple¬ 
tion  of  a  forward  flip,  sideways  roll,  tail-in  funnel  and  nose-in  funnel  for  a  helicopter. 
They  first  used  a  pilot  to  fly  the  helicopter  to  generate  a  dynamics  model  and  cost 
function,  then  applied  the  apprenticeship  learning  algorithm  to  develop  a  near-optimal 
policy  to  perform  the  difficult  non-linear  aerobatic  maneuvers.  This  state-of-the-art 
work  in  autonomous  helicopter  flight  presents  an  intriguing  approach  to  the  non¬ 
linear  transitions  required  to  navigate  between  thrust-borne  and  wing-borne  flight  for 
an  agile  autonomous  aircraft  and  is  a  step  above  and  beyond  simple  open-loop  control 
sequences. 

1.4  Contribution 

The  main  contributions  of  this  work  are  as  follows.  Thrust-borne  controllers  are  devel¬ 
oped  for  the  YAK-54  and  Aerovel  Flexrotor  aircraft  via  linearization  and  decoupling 
of  the  resulting  linear  systems.  Linear  quadratic  control  techniques  are  used,  with 
the  addition  of  integral  terms  in  certain  cases.  Control  methods  for  autonomous  tran¬ 
sitions  between  wing-borne  and  thrust-borne  flight  regimes  are  also  discussed.  The 
problem  of  reduced  order  control  is  contemplated  for  the  YAK-54  with  recommenda¬ 
tions  for  a  sensing/control  suite  to  be  used  for  hardware  implementation  and  flight 
testing. 

Additionally,  a  general  framework  for  control  of  agile  aircraft  capable  of  sustained 
autonomous  hover  is  proposed  based  on  the  developed  controllers.  Simulation  results 
indicate  the  satisfactory  performance  of  the  LQ  control  laws  for  thrust-borne  flight 
and  validate  the  proposed  framework. 

Throughout  this  work,  noise-free  sensing  and  full  state  feedback  are  assumed, 
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unless  otherwise  noted.  Also  assumed  is  a  disturbance  free  environment  (no  wind  in 
simulations). 

1.5  Thesis  Organization 

This  thesis  is  organized  as  follows.  Mathematical  preliminaries  necessary  for  the  un¬ 
derstanding  of  theoretical  contributions  of  this  thesis  are  detailed  in  Chapter  2.  Next, 
Chapter  3  contains  the  theoretical  contributions  of  this  work  in  detail.  Simulation 
results  supporting  the  theory  are  presented  in  Chapter  4.  Finally,  Chapter  5  contains 
concluding  remarks. 
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Chapter  2 

MATHEMATICAL  PRELIMINARIES 


The  work  here  is  an  instance  of  established  theory  in  vehicle  control  techniques. 
Mathematical  details  of  previous  work  and  mathematical  techniques  in  use  throughout 
this  thesis  are  the  topic  of  this  chapter. 

2.1  Linear  Systems  Theory 

The  motion  and  control  of  agile  aircraft  is  dictated  by  the  aircraft’s  nonlinear  dynam¬ 
ics,  described  by  equations  of  the  form 


Xi  =  fi{x  1,.. 

•  t  D  ^1 5  •  • 

•  )  U"m)  ?  i  !)•••)  ^5 

(2.1) 

Vi  =  9i(x  i,.. 

•  5  ^1 1  •  • 

•  5  ^m)  •>  i  1?  •  •  •  j  Vi 

(2.2) 

where  ui7  i  —  1, . . . ,  m,  denotes  the  system  inputs;  yi:  i  —  1, . . .  ,p,  denote  the  system 
outputs;  Xi,  i  =  1, . . .  ,n,  denote  the  system  states;  x%  denotes  the  time  derivative  of 
the  states,  x^,  and  /*  and  are  real- valued  functions  [5].  Time  is  not  explicitly  present 
in  equations  (2.1)  and  (2.2)  as  the  aircraft  dynamics  are  assumed  be  time-invariant. 
The  system  dynamics  can  be  represented  in  vector  form  as 


f{x,u), 

(2.3) 

g(x,  u), 

(2.4) 

where  x  E  IFT  is  the  state  vector,  u  E  is  the  input  vector,  and  y  E  IFF  is  the  output 
vector  [5]. 

Nonlinear  system  dynamics  can  then  be  approximated  as  linear  around  equilib¬ 
rium  or  trim  solutions,  such  as  wings-level  flight  or  hover  for  an  agile  aircraft.  The 
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approximations  are  linear  as  the  state  derivatives  change  according  to  a  linear  func¬ 
tion  of  the  states  and  controls.  Whether  the  linear  dynamics  are  found  via  numerical 
linearization  or  through  a  Taylor  Series  approximation,  the  linear  dynamics  have  the 
form 


x  =  Ax  +  Bu,  (2.5) 

y  =  Cx  +  Du,  (2.6) 

where  A  G  IRnxn,  B  G  K'lXm,  C  G  IRpxn,  and  D  G  IRpXm. 

Approximating  nonlinear  systems  as  linear  allows  the  use  of  linear  control  tech¬ 
niques,  such  as  optimal  feedback  control.  Additionally,  the  simplicity  provided  by 
linearization  combined  with  agile  aircraft  likely  to  spend  the  majority  of  missions  in 
linear  flight  regimes  makes  it  a  powerful  and  useful  tool. 

2.2  Optimal  Feedback  Control 

In  order  to  satisfy  the  mission  requirements  or  goals  for  any  agile  aircraft  system,  some 
form  of  control  signals  are  required.  When  the  aircraft  is  operating  in  a  flight  regime 
in  which  a  linear  approximation  of  the  dynamics  is  valid,  optimal  feedback  control 
signals  can  be  generated  according  to  the  theory  of  the  Linear-Quadratic-Regulator 
(LQR)  problem  or  the  Linear-Quadratic-Gaussian  (LQG)  problem.  Linear  quadratic 
approaches  are  so  called  since  the  system  of  dynamics  of  concern  are  linear,  while  the 
performance  measure  involves  the  square  of  a  signal,  hence  quadratic  [7]. 

For  a  deterministic  linear  system  of  the  form  (2.5)-(2.6),  a  control  law  of  the  form 
u  =  —  Kx  that  minimizes  the  performance  measure 

V=  [T(x'Qx  +  u'Ru)dt  +  x'(T)Mx(T)  (2.7) 

Jo 

is  desired.  In  this  formulation  of  the  LQR  problem,  Q  and  M  are  typically  positive- 
semidehnite  matrices,  and  R  is  a  positive-definite  matrix  [7].  The  matrix  Riccati 
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equation1  and  final  boundary  value  for  the  symmetric  matrix  P(t) 

-P  =  A'P  +  PA  +  Q-  PBR~lB'P ; 

P(T)  =  M  (2.8) 

lead  to  the  optimal  state-feedback  control  given  by 

u  =  — K(t)x ; 

K{t)  =  R~lB'P{t).  (2.9) 

The  LQG  problem  is  an  extension  of  the  linear  quadratic  approach  to  linear 
stochastic  systems  of  the  form 

x  =  Ax  +  Bu  +  £  (2.10) 

y  =  Cx  +  e  (2.11) 

where  £(t)  and  8(t)  are  uncorrelated  zero-mean,  Gaussian,  white-noise  random  vectors 
with  correlation  matrices 


E{mPm  =  E5{t  -  r);  E{9it)e\t)}  =  Q6(t  -  r).  (2.12) 

The  random  vector  £(t)  is  referred  to  as  the  process  noise,  and  9(t)  is  referred  to  as 
the  measurement  noise.  Finding  a  control  u  that  minimizes  the  cost  function 


V  =  lim  E{x'Qx  +  u'Ru} 

t—>  OO 


(2.13) 


represents  the  LQG  problem. 


lrThe  formulation  of  the  Riccati  equation  and  a  discussion  of  the  Hamilton- Jacobi  optimization 
equation  has  been  omitted.  The  information  given  here  is  sufficient  for  understanding  of  the 
remainder  of  this  thesis,  but  interested  readers  may  wish  to  refer  to  Dorato  [7]  for  more  detailed 
information. 
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2.3  Open  Loop  Control 

While  LQ  techniques  can  be  used  for  feedback  control  of  flight  regimes  approximated 
as  linear,  the  transitions  between  linear  flight  regimes  require  a  different  approach. 
The  simplest  form  of  open  loop  control  is  a  predetermined  signal  designed  by  the 
system  operator  or  creator.  In  order  to  improve  on  the  basic  open  loop  control 
sequences,  machine  learning  schemes  can  be  used. 

2.3.1  Markov  Decision  Processes 

A  Markov  decision  process  (or  MDP)  is  the  specification  of  a  sequential  decision 
problem  for  a  fully  observable  environment  with  a  Markovian  transition  model  and 
additive  rewards  [23].  An  MDP  is  defined  by  three  components:  an  initial  state 
So,  a  transition  model  T(s,a,s'),  and  a  reward  function  R(s).  The  transition  model 
denotes  the  probability  of  reaching  the  state  s'  if  action  a  is  executed  in  state  s.  The 
assumption  of  a  Markovian  transition  model  implies  that  the  probability  of  reaching 
s'  from  s  depends  only  on  s  and  not  on  any  earlier  states  visited  by  the  system. 

A  solution  to  an  MDP  is  termed  a  policy  n,  which  specifies  what  actions  to  take 
for  any  state  that  the  system  may  reach.  The  action  specified  by  a  policy  n  for  a 
system  at  state  s  is  depicted  as  7r(s).  When  executing  a  policy,  the  sum  of  the  rewards 
for  all  the  visited  states  is  the  utility  of  that  policy.  Given  the  stochastic  nature  of  a 
system  and  its  environment,  repeatedly  executing  a  policy  from  an  initial  condition 
leads  to  different  trajectories  and  different  utilities.  The  expected  utility  of  a  policy 
can  then  be  thought  of  as  a  measure  of  the  quality  of  a  policy  [23].  The  optimal 
policy,  denoted  7 r*,  is  the  policy  that  yields  the  highest  expected  utility  for  an  MDP. 

2.3.2  Reinforcement/ Apprenticeship  Learning 

Reinforcement  learning  is  a  general  category  of  machine  learning  in  which  feedback 
in  the  form  of  rewards  or  reinforcement  is  used  to  improve  the  performance  of  a 
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system.  Reinforcement  learning  problems  also  include  the  subproblem  of  learning  how 
the  environment  works.  Numerous  algorithms  are  available  to  find  optimal  policies 
for  solving  MDPs  that  include  value  iteration,  policy  iteration,  adaptive  dynamic 
programming,  temporal  difference  schemes,  Q-learning  agent,  policy  search,  and  the 
E3  algorithm. 

An  alternative  approach  to  the  reinforcement  learning  problem  developed  in  [1] 
is  that  of  apprenticeship  learning.  Whereas  reinforcement  learning  algorithms  focus 
on  a  combination  of  exploitation  or  exploration  of  the  state  space,  apprenticeship 
learning  focuses  on  exploiting  regions  of  the  state  space  near  a  policy  provided  by  a 
teacher.  The  algorithm  works  to  find  a  near-optimal  policy  based  on  the  teachers  that 
avoids  dangerous  parts  of  the  state  space.  The  apprenticeship  learning  algorithm  is: 

1.  Run  N  trials  under  the  teacher’s  policy  ht  of  system  M.  Save  the  state-action 

trajectories  of  these  trials  and  compute  an  estimate  of  the  utility  of 

the  teacher’s  policy,  by  averaging  the  sum  of  rewards  in  each  trial. 

2.  Using  all  state-action  trajectories  collected,  estimate  the  system  dynamics  T 
using  regularized  linear  regression  for  the  linearly  parameterized  dynamics  case. 
Call  the  estimated  dynamics  T^\ 

3.  Find  an  a/8  (where  a  >  0)  optimal  policy  for  the  MDP  M W  =  (S,  A,  T^\  H,  D,  R ), 
and  call  this  policy 

4.  Evaluate  the  utility  of  policy  on  real  system  M  for  k  trials.  Save  all  state- 

action  trajectories,  and  let  be  the  average  sum  of  rewards  accumulated. 

5.  If  Um( 7r^)  >  Um(kt)  ~  oi/ 2,  return  otherwise  set  i  —  i  +  1,  and  go  back 
to  step  2  [1], 
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2.4  Aircraft  Orientation 

In  modeling  the  six  degrees  of  freedom  of  an  aircraft  in  flight,  the  aircraft  orientation 
and  position  must  be  oriented  to  some  fixed  frame,  traditionally  the  vehicle-carried 
Earth  axis  system.  The  common  approach  to  depicting  an  aircraft’s  orientation  is 
to  use  Euler  angles,  an  ordered  set  of  rotations;  however,  Euler  angles  present  a  few 
problems  if  the  system  is  capable  of  sustained  vertical  flight.  As  a  result,  alternate 
methods  of  depicting  the  aircraft  orientation  must  be  considered  and  used  to  replace 
or  supplement  the  Euler  angle  depictions  of  attitude. 


2-4-1  Euler  Angles 

The  order  dependent  Euler  angle  transformation  starts  from  the  standard  North- East- 
Down  (NED)  axes  and  translates  the  aircraft’s  orientation  to  the  body  axes.  First,  a 
right-handed  yaw  rotation  (0)  is  performed  about  the  z-axis.  Second,  a  right-handed 
pitch  rotation  (9)  is  performed  about  the  new  position  of  the  y- axis.  Finally,  a  right- 
handed  roll  rotation  (0)  is  performed  about  the  new  position  of  the  x-axis  [25]. 

The  plane  rotation  matrices  can  be  written  according  to  the  prescribed  rules  of 
the  rotations  as 


1 

0 

0 

c6 

0 

-sO 

Cl l) 

sijj 

0 

0 

C0 

S0 

0 

1 

0 

— S0 

Cl\) 

0 

0 

—  S(f) 

C(j) 

s9 

0 

c6 

0 

0 

1 

(2.14) 


where  ub  is  a  vector  expressed  in  the  body  frame,  and  ur  is  the  vector  expressed  in 
the  reference  frame.  Note  also  that  an  abbreviating  convention  is  used  where  cosine 
and  sine  are  shortened  to  c  and  s,  respectively.  The  three  rotation  matrices  can  by 
multiplied  together  to  form  an  invertible  transformation  matrix  from  the  reference 
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system  to  the  body  system  as 

cdci/j  cdsip  —s6 

Cb/r  =  —cQsi\)  +  s^sdc'ijj  c0c0  +  srfisdsip  s(f>c9  ■  (2-15) 

s0s0  +  c(j)s9cil>  — s(f>cip  +  c(ps9s 0  c<j)c9 

Problems  arise  with  Euler  angle  determination  of  aircraft  orientation  when  the 
the  pitch  angle  (9)  is  90°,  as  the  roll  and  heading  angles  are  not  uniquely  determined. 
Additionally,  wrap-around  difficulties  present  themselves  when  the  yaw  and  roll  angles 
exceed  their  bounds  of  ±180°  [6]. 

The  kinematic  relationships  for  rotation2  can  be  found  by  letting  the  Euler  angles 
have  time  derivatives  (0,0,0),  and  using  the  components  of  the  aircraft’s  angular- 
velocity  vector, 

0  1  t9scf>  t9ccj)  P 

9  =  0  c0  — s0  Q  (2-16) 

0  0  s(j)/c9  c(f>/c6  R 

where  P,  Q ,  R  are  the  standard  symbols  for  roll,  pitch,  and  yaw  rate  of  an  aircraft 

[25]. 

2-4-2  Vertical  Euler  Angles 

One  approach  to  avoiding  the  deficiencies  of  the  Euler  angles  is  to  develop  a  second  set 
of  rotations,  termed  the  vertical  Euler  angles.  This  second  set  of  angles  can  be  used 
to  provide  unambiguous  attitude  depiction  while  aircraft  are  operating  in  a  vertical 
flight  regime.  As  with  the  Euler  angles,  the  vertical  Euler  angles  are  an  ordered  set 
of  right-handed  rotations.  Starting  with  the  aircraft  in  a  vertical  attitude  with  the 
body  rc-axis  pointing  opposite  the  NED  z-a.xis  and  the  body  z-a.xis  pointing  North, 
aligned  with  the  NED  rc-axis,  perform  a  roll  rotation  (0„)  about  the  body  x-axis. 

2A  complete  derivation  of  this  kinematic  relationship  is  omitted  for  succinctness.  Please  see  [25] 
for  more  information. 
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Next,  perform  a  pitch  rotation  (0V)  about  the  new  ?/-axis.  Finally,  perform  a  yaw 
rotation  (' 0„ )  about  the  twice  moved  z-axis,  as  seen  in  Figure  (2.1). 


Figure  2.1:  Depiction  of  the  rotations  of  the  vertical  Euler  method  of  tracking  orien¬ 
tation  [6]. 


In  the  vertical  Euler  angle  system,  the  order  of  rotations  is  roll,  pitch,  yaw;  the 
opposite  of  the  yaw,  pitch,  roll  rotations  of  the  traditional  Euler  angle  rotations. 
Additionally,  since  the  same  idea  is  applied  to  rotate  the  axes  in  both  sets  of  rotations, 
singularities  are  present  in  the  vertical  Euler  angle  depiction  as  well.  Here,  the  vertical 
roll  and  yaw  angles  are  not  uniquely  determined  when  the  vertical  pitch  angle  is  90°, 
when  the  aircraft  is  in  level  horizontal  flight. 

The  vertical  Euler  angles  allow  intuitive  angular  representations  of  an  aircraft’s 
orientation  in  the  vertical  regimes  with  flight,  while  traditional  Euler  angles  pro¬ 
vide  the  same  intuitive  representation  in  horizontal  flight  regimes.  By  using  both  in 
conjunction  with  one  another,  attitude  representations  can  be  made  throughout  the 
whole  spectrum  of  flight;  however,  this  approach  complicates  calculation  of  the  time 
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derivatives  of  the  Euler  angles. 


2.4.3  Quaternions 


In  order  to  avoid  the  singularity  and  computational  issues  of  the  Euler  angle  approach, 
quaternions  can  be  used.  Though  much  less  intuitive,  the  quaternion  approach  is 
more  robust,  and  still  allows  the  use  of  Euler  angles  for  visualization  of  the  aircraft’s 
orientation.  Quaternions,  of  the  form  x0+xii+x2j +x3k  allow  an  alternate  expression 
of  the  kinematical  equations: 


qo 

0  -P  -Q  -R 

qo 

q\ 

1 

P  0  R  -Q 

qi 

q2 

”  2 

Cb 

0 

1 

Q> 

q-2 

qs  _ 

O 

Cb 

1 

Q> 

Qb 

.  ^  . 

(2.17) 


For  compound  rotations  involving  a  combination  of  pitch,  roll,  and  yaw,  the  quater¬ 
nion  rotation  axis  is  not  evident  [25],  so  quaternions  are  often  initialized  from  Euler 
angles.  Further,  while  the  orientation  is  tracked  via  the  quaternion,  it  is  possible  to 
obtain  the  Euler  angles  (or  vertical  Euler  angles)  as  needed  or  desired.  The  relation¬ 
ship  between  the  quaternions  and  Euler  angles  is  given  by 


q0  =  cos(0/2)  cos(0/2)  cos(^/2)  +  sin(0/2)  sin(6*/2)  sin('?/i/2)  (2.18) 

qi  =  sin(</>/2)  cos(0/2)  cos(?/i/2)  —  cos(0/2)  sin(#/2)  sin(-0 /2)  (2.19) 

q-2  =  cos(0/2)  sin(6>/2)  cos(^/2)  +  sin(0/2)  cos(6>/2)  sin(^/2)  (2.20) 

q3  =  cos(0/2)  008(0/2)  sin(^/2)  —  sin(0/2)  sin(0/2)  cos(?/i/2),  (2.21) 


and  the  relationship  between  the  quaternions  and  vertical  Euler  angles  is  given  by 

(2.22) 


qo  = 


q\  = 


'1-sin  (6V)  (^v-(t>v 

- z - cos 


1  +  sin(di; 


sm 


V  2 

'Ipv  *Pv 


(2.23) 
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9*  =  (2.24) 

93  =  f ^fWsill  (AxA)  .  (2.25) 

To  obtain  the  Euler  or  vertical  Euler  angles  from  quaternions,  first  the  transformation 
matrix  is  formed, 


B 


Qo  +  ql  -  ql  -  q! 

2(<M2  -  <?0<?3 ) 

2(<?i<?3  +  qoq2) 


2(<?i<?2  +  9o93) 

<?02  -  ql  +  <?22  -  %2 

2(<M3  -  9o9i) 


2(<m3  -  <?o<?2) 
2(g2<?3  +  <Mi) 

ql  -  ql  -  ql  +  ql 


(2.26) 


the  elements  bij  of  which  are  used  to  End  the  Euler  angles: 


0 

tan  x®) 

e 

= 

-sin_1(613) 

*l> 

tan_1(|f) 

or  the  vertical  Euler  angles: 


“tan"1®) 

9V 

= 

sin-1  (633) 

A 

tan”1®) 

(2.27) 


(2.28) 


This  approach  of  internally  calculating  the  quaternions  in  the  aircraft  dynamics, 
but  using  a  combination  of  the  normal  Euler  angles  and  the  vertical  Euler  angles  to 
present  vehicle  orientation  is  used  for  the  modeling  of  the  Tandem-Wing  Tail-Sitter 
UAV  [6] ,  in  which  Euler  angles  are  used  for  pitch  angles  less  than  45°  and  the  vertical 
Euler  angles  are  used  for  pitch  angles  greater  than  45°. 


2-4-4  Incremental  Rotations 

A  final  method  of  depicting  the  orientation  of  the  aircraft  is  to  use  incremental  rota¬ 
tions.  In  the  case  of  the  Aerovel  Flexrotor,  (f>x,  (py,  and  (f>z  are  incremental  rotations 
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about  the  respective  body  axes.  In  other  words,  they  are  values  resulting  from  the 
integration  of  the  respective  rotation  rates  of  the  Flexrotor  dynamics  [18]. 

This  simpler  method  of  tracking  and  presenting  orientation  can  be  derived  from 
equation  (2.16)  through  the  small  angle  approximation.  If  0,  0,  and  0  are  all  close 
to  zero,  then  the  cosines  are  approximately  1  and  the  sines  are  approximately  the 
angular  values,  or  zero,  so  equation  (2.16)  becomes 

0  10  0  P 

0  =  0  1  0  Q  ■  (2.29) 

0  0  0  1  R 

The  task  of  tracking  and  presenting  aircraft  orientation  is  complicated  by  studying 
an  agile-aircraft  capable  of  sustained  vertical  flight;  however,  by  combining  the  various 
approaches  here,  full  and  accurate  attitude  information  is  available. 

2.5  Wing-Borne  Flight 


Aircraft  dynamics  are  prescribed  by  the  general  six  degree  of  freedom  formulation, 
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where  the  forces,  Fx ,  Fy,  Fz,  moments,  L,  M ,  IV,  and  the  moments  and  products 
of  inertia  Ixx,  Iyy,  Ixz,  Izz  are  specihc  to  each  aircraft  [25],  and  the  transformation 
matrix  B  is  given  by  Equation  2.26.  In  this  formulation  (Equation  2.32),  the  Euler 
kinematic  equation  is  used,  though  the  quaternion  implementation  could  be  used  in 
its  place. 

In  wing-borne  flight,  the  wings  provide  lift  to  balance  the  force  of  gravity,  while 
the  motor  provides  a  propulsive  force.  In  a  trimmed,  wings-level  unaccelerated  flight 
condition,  the  four  forces  of  flight  balance  out  so  that  L  =  W  and  T  =  D. 

2.6  Thrust-Borne  Flight 

Aircraft  capable  of  sustained  vertical  flight,  or  thrust-borne  flight,  present  a  unique 
set  of  challenges  when  compared  to  traditional  aircraft.  Consider  an  agile  platform 
in  a  trimmed  hover  configuration,  so  that  thrust  equals  weight  ( T  =  W),  and  lift 
and  drag  forces  are  reduced  to  magnitudes  near  zero  and  can  be  ignored.  While  the 
6-DOF  equations  (2.30,  2.31,  2.32,  and  2.33)  still  apply,  special  attention  needs  to  be 
made  to  the  modeling  of  the  aircraft.  In  a  hover  configuration,  the  throttle  is  used  to 
control  altitude,  ailerons  control  the  vertical  roll  rate,  the  rudder  controls  the  velocity 
in  the  body-axis  y  direction,  and  the  elevator  controls  the  velocity  in  the  body-axis  z 
direction. 

Typically,  to  simplify  modeling,  control  surface  contributions  to  aircraft  dynamics 
are  a  function  of  the  free  stream  velocity,  but  in  hover,  the  free  stream  velocity  goes  to 
zero.  To  accurately  model  the  control  surfaces,  the  prop  wash  must  be  modeled,  as  it 
provides  necessary  airflow  over  the  ailerons,  rudder,  and  elevator  to  allow  control  while 
in  hover.  Some  aircraft  are  further  restricted  in  their  controllability  if  the  ailerons  are 
out-board  of  the  prop  wash.  The  rotational  effects  of  the  prop  wash  and  engine  must 
also  be  taken  into  account,  and  balanced  by  the  ailerons  (or  some  other  means)  while 
in  a  stabilized  hover. 
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Chapter  3 

SYSTEM  MODELING  AND  CONTROLLER  DESIGN 

The  development  of  various  controllers  for  the  YAK-54  and  the  Aerovel  Flexrotor 
is  the  focus  of  this  chapter.  Initially,  each  system  is  described,  including  the  derivation 
of  system  dynamics  and  linearization.  Based  on  the  linearization  of  each  system,  linear 
controllers  will  be  considered  for  thrust-borne  and  wing-borne  flight  regimes.  Finally, 
open  loop  controllers  will  be  proposed. 

3.1  System  Description 

The  two  aircraft  under  consideration  here  are  the  YAK-54  and  the  Aerovel  Flexrotor. 
While  both  have  thrust  to  weight  ratios  in  excess  of  one,  they  vary  significantly  in 
terms  of  design  and  performance.  The  YAK-54  is  a  highly  agile  aircraft  capable  of 
complex  aerobatic  maneuvers.  The  Flexrotor  is  designed  for  long  endurance  flight  and 
presents  a  different  approach  to  thrust-borne  flight  through  the  use  of  tip  thrusters 
for  roll  control. 

3.1.1  YAK-54 

The  Yakovlev  YAK-54  is  a  Russian  built  aerobatic  aircraft  that  first  flew  on  the  23rd 
of  December,  1993  [29].  Scale  models  to  the  YAK-54  have  enjoyed  prominence  in 
the  remote  control  aircraft  world  for  their  exceptional  maneuverability  and  ability  to 
perform  complex  3D  acrobatics. 

The  YAK-54  studied  here  is  a  69  inch  almost-ready-to-fly  (ARF)  remote  control 
aircraft  manufactured  by  NitroModels  (see  Figure  3.1).  While  this  size  of  RC  aircraft 
typically  employs  small  two  stroke  gas  engines  in  the  1.0  to  2.0  cubic  centimeter  size, 
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an  electric  motor  was  selected.  The  Turnigy  63-64-B  motor  provides  a  high  level  of 
reliability  and  is  easier  to  install  and  operate.  The  motor,  combined  with  a  20  inch 
diameter  prop  with  10  inches  of  pitch,  provides  the  YAK-54  with  a  thrust  to  weight 
ratio  greater  than  one.  The  significant  limitation  of  the  electric  motor  is  a  short 
endurance  -  roughly  five  to  ten  minutes  depending  on  throttle  settings  throughout 
the  flight. 

3.1.2  Aerovel  Flexrotor 

The  Aerovel  Flexrotor  was  designed  with  long  range,  long  endurance,  and  a  vertical 
take  off  and  landing  (VTOL)  capability.  Similar  in  design  and  concept  to  tail-sitter 
aircraft  developed  and  tested  during  the  1950s,  such  as  the  Convair  XFY-1  Pogo, 
Lockheed  XFV-1  Salmon,  and  the  Ryan  X-13A  Vertijet  [22],  the  Flexrotor  concept 
has  these  distinctive  features  and  design  goals: 

•  a  large  hybrid  propeller /rotor  with  low  disc  loading, 

•  a  variable-pitch  rotor  hub, 

•  the  capability  to  perform  stationary  launch  in  a  vertically  oriented  position, 

•  a  high  speed  cruise  in  a  typical  aircraft  configuration, 

•  standard  control  surfaces  in  wing-borne  flight, 

•  a  unique  combination  of  a  rotor  cyclic  and  wing-tip  thrusters  for  control  in 
thrust-borne  flight,  and 

•  the  capability  to  recover  autonomously  from  thrust-borne  flight  [19]. 

The  ability  to  launch  and  recover  from  a  vertical  orientation  is  significantly  different 
from  that  of  the  YAK-54,  which  can  only  takeoff  and  land  in  a  conventional  manner. 
Figure  1.3  shows  the  general  layout  of  the  Flexrotor,  including  the  tractor  propeller 
and  the  wing-tip  thrusters,  along  with  a  small  V-tail  for  rudder  and  elevator  control. 


24 


Figure  3.1:  A  three- view  of  the  YAK-54  showing  all  geometry  and  dimensions  (in 
mm)  [15]. 
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3.2  System  Dynamics  and  Modeling 

Just  as  both  aircraft  are  different  in  terms  of  design,  both  are  modeled  in  different 
manners.  Though  the  basic  dynamics  of  any  aircraft  are  quite  similar,  the  modeling 
and  specifics  of  the  dynamics  vary  significantly.  The  YAK-54  is  a  conventional  aircraft 
in  design,  and  can  be  easily  modeled  for  wing-borne  flight;  however,  flight  near  vertical 
or  in  unusual  angles  of  attack  presents  some  difficulties.  The  dynamics  of  the  YAK- 
541  are  modeled  in  a  Matlab/Simulink  environment  that  makes  use  of  the  Aerosim 
Blockset.  Other  modeling  challenges  include  the  lack  of  airspeed  over  the  wings  while 
in  thrust-borne  flight.  As  the  only  airflow  is  provided  by  the  propeller,  prop  wash 
and  its  effects  must  be  accounted  for.  The  Flexrotor  dynamics2  are  complicated  by 
the  helicopter  style  prop  and  the  use  of  tip  thrusters  for  roll  control.  The  dynamics 
of  the  Flexrotor  are  modeled  in  a  proprietary  software  package  developed  by  Aerovel. 

3.2.1  YAK-  54 

The  state  vector  for  the  Yak-54  is 

x  —  [  u  v  w  p  q  r  PN  PE  Pd  <t>  8  }T  (3-1) 

or 

x  =  [  u  v  w  p  q  r  PN  PE  Pd  <t>v  4>v  Y  (3-2) 

depending  on  the  current  flight  regime,  where  u,  v,  w  represent  the  aircraft  velocity, 
p,  q,  r  represent  the  aircraft  roll,  pitch,  and  yaw  rates,  and  P/v,  PE,  Pd  represent 
the  aircraft’s  position  in  the  NED  frame.  In  wing-borne  flight,  Euler  angles  are  used, 
whereas  in  thrust-borne  flight  the  vertical  Euler  angles  are  used  for  position  tracking. 

1For  a  complete  dialogue  on  the  derivation  of  the  YAK-54’s  modeling  and  dynamics,  please  see 

[15]- 

2For  more  information  regarding  the  dynamics,  please  see  [19],  [20],  and  [17]  and  for  more  infor¬ 
mation  regarding  the  proprietary  nature  of  the  modeling  and  dynamics  of  the  Aerovel  Flexrotor 
contact  Dr.  Tad  McGeer  of  Aerovel. 
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The  input  vector  of  controls  is 

u=[ST  Se  8r  8a  f,  (3.3) 

where  Sr  is  the  throttle  input  and  Se,  Sr,  Sa  are  the  elevator,  rudder,  and  aileron 

inputs,  respectively. 

The  aileron  range  has  limits  of  ±35  degrees,  while  the  elevator  and  rudder  both 
have  ranges  of  ±45  degrees.  The  throttle  fractional  varies  between  0  and  1,  with  0 
corresponding  to  a  fully  closed  throttle,  and  1  corresponding  to  maximum  throttle. 

3.2.2  Aerovel  Flexrotor 

The  state  vector  for  the  Flexrotor  is 

X  =  [  U  v  w  p  q  r  4>x  (j)y  <pz  PN  PE  Pd  V  (3-4) 

where  u,  v,  w,  p ,  q ,  r,  Pn,  Pe,  and  Pd  represent  the  same  variables  as  in  the  YAK- 

54  dynamics,  and  orientation  is  tracked  through  the  incremental  rotations  <f>x ,  (f)y , 
and  (j)z.  The  Flexrotor’s  input  vector  is 

U=[  P0  thr  frac  R0  hub  beta  dp' c  dq' c  dr' c  ]T  (3.5) 

where  P0  thr  frac  is  the  throttle  fractional,  R0  hub  beta  is  the  collective  pitch  angle  for 
the  main  rotor,  and  dp'c ,  dq'c,  and  dr'c  are  diagonalized  control  inputs.  A  mixing 
matrix  then  specifies  the  map  from  the  diagonalized  inputs  to  the  physical  controls 
[18].  The  remaining  physical  controls  are  the  aileron,  elevator,  and  rudder  inputs: 
Sa,  Se,  Sr,  the  cyclic  settings  of  the  main  rotor:  R0  cyc  y,  R0  cyc  z,  and  the  power 
settings  for  the  tip  thrusters:  P\  power,  ±2  power- 

3.3  Linearization 

Both  the  YAK-54  and  the  Flexrotor  make  use  of  numerical  linearization  techniques 
to  find  trim  conditions  and  the  matrices  necessary  to  define  an  LT1  system. 
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3.3.1  YAK-  54 


For  thrust-borne  flight  using  the  vertical  Euler  angles  and  neglecting  the  navigation 
equations,  the  YAK-54  dynamics  are  given  by  [15]: 
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In  order  to  linearize  the  equations  of  motion  in  thrust-borne  flight  around  a  given 
vertical  velocity  u \ ,  substitutions  are  made  to  the  aircraft  states: 
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After  applying  standard  small  angle  assumptions  and  discarding  second  order  terms, 
the  equations  of  motion  (3.6)-(3.14)  of  the  YAK-54  then  become: 


u 

MFx 

—  —g  h — 

m 

(3.35) 

V 

i  ,  |  Fv 

=  -r  U\  +  gipv  +  — 

m 

(3.36) 

w 

a  Fz 

=  q  u\  -  g9v  H - 

m 

(3.37) 

P 

=  C3L  +  C4  N  =  L/ Ixx 

(3.38) 

q 

=  C7M  =  M/Iyy 

(3.39) 

r 

—  cwL  +  C11 A  =  A /Izz 

(3.40) 

4*v 

=  P 

(3.41) 

29 


9V  =  q  (3.42) 

i>v  =  r  (3.43) 

where  instead  of  A p,  we  have  written  p.  Additionally,  the  product  of  inertia  Ixz  is 
taken  to  be  zero  as  it  is  assumed  to  be  significantly  smaller  than  the  moments  of 
inertia  [15]. 

As  the  force  and  moment  terms  are  functions  of  the  states  and  inputs,  a  first 
order  Taylor  expansion  is  used.  For  example,  the  Taylor  expansion  of  the  forces  in 
the  body-axis  x  direction  is  given  by 

Fx  =  FXl  +  FXu  Au  +  FXv  Av  +  FXwAw  +  FXpAp  +  FXqAq  +  FXrAr 

+  FXStA5t  +  FXSeA5e  +  FXSr  A5r  +  FXSa  ASa,  (3.44) 

where  the  second  subscript  denotes  the  partial  derivative  with  respect  to  that  variable 
( FXu  =  [6].  In  order  for  u  to  limit  to  zero  in  Equation  3.35,  the  zero-order  term 

of  Equation  3.44  must  be  equal  to  mg,  while  the  zero-order  terms  of  Fy,  Fz,  L,  M , 
and  N  are  all  zero. 

Following  the  analysis  of  [15],  many  of  the  partial  derivatives  of  each  Taylor  ex¬ 
pansion  can  reasonably  be  expected  to  be  close  to  zero  for  the  thrust-borne  flight 
regime,  so  the  LTI  system  is  given  by 
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where  the  navigation  equations  are  included,  assuming  that  the  trim  condition  is  with 
the  body-axis  z  direction  facing  north,  so  that  the  vertical  Euler  angles  are  zero. 

With  Equation  3.45  in  the  form  of  Equation  2.5,  the  LTI  system  is  transformed 
in  order  to  reorder  the  states  and  partition  the  dynamics  from  one  large  system  into 
four  smaller  decoupled  systems.  The  transformation  matrix 
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is  used  to  form  the  new  state-space  system 


z  =  Az  +  Bu 


(3.47) 


where  z  =  T  1x,  A  =  T  lAT  and  B  =  T  lB.  This  state  transformation  admits  the 
following  decoupled  subsystems. 


(3.45) 
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The  axial  subsystem  with  xa  =  [u  h]T  and  ua  =  5T  is  given  by: 
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The  roll  subsystem  with  xr  =  [p  (pv]T  and  ur  =  5a  is  given  by: 
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The  longitudinal  subsystem  with  xiong  —  [w  q  9V  Pn}t  and  uiong  =  5e  is  given 
by: 
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The  lateral  subsystem  with  xiat  =  [u  r  %j)v  Pe]t  and  uiat  =  Sr  is  given  by: 
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For  the  aircraft  in  a  vertical  position  with  the  belly  pointing  north  (body-axis  z 
direction)  with  zero  vertical  velocity  U\ ,  the  trim  inputs  are 


u0  =  [  0.7396  0  -0.0961  0  ]T, 


(3.52) 


which  have  the  expected  form  of  a  non-zero  throttle  input  to  counteract  weight  and  a 
non- zero  aileron  input  to  counteract  the  rotational  effects  of  the  motor  and  the  prop 
wash. 
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3.3.2  Aerovel  Flexrotor 

The  linearization  of  the  Flexrotor  in  thrust-borne  flight  is  performed  by  Aerovel’s 
comprehensive  software  suite  developed  for  the  Flexrotor.  The  nonlinear  dynamics 
prescribed  by  (2.30)  -  (2.33)  are  then  expressed  as  an  LTf  system  in  the  form  of 
(2.5).  As  with  the  YAK-54,  a  state  transformation  (3.46)  is  applied  to  diagonalize 
the  system  and  decouple  it  into  subsystems. 

•  The  axial  subsystem  with  xa  =  [u  h]T  and  ua  =  [To  Ro]1  is  given  by: 


-0.41  0  u 

— 1  0  h 


0.54  -1.6 
0  0 


(3.53) 


The  roll  subsystem  with  xr  =  [p  (j)v]T  and  ur  =  dp' c  is  given  by: 

-1.4  0  1  [  p  1  [  1  1 

xr  —  +  dp  c. 

10  (b„  0 


(3.54) 


The  longitudinal  subsystem  with  Xiong  =  [w  q  9V  Pn]T  and  Uiong  =  dq'c  is  given 


-0.47  0.05  -9.81  0 


0.32  -2.27 


long 


0  0 
0  0 


0  0 


(3.55) 


The  lateral  subsystem  with  xiat  =  [v  r  i\)v  PE]T  and  uiat  =  dr'c  is  given  by: 


% lat. 


-0.22  9.81  0 

-2.31  0  0 

1  0  0 

0  0  0 


(3.56) 
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Due  to  the  use  of  the  diagonalized  control  inputs,  all  three  controls  for  the  axial, 
longitudinal,  and  lateral  subsystems  of  the  Flexrotor  must  be  mapped  back  to  the 
physical  controls  by  using  the  prescribed  mixing  matrix  for  the  linearization.  For 
thrust-borne  flight,  the  physical  controls  are  given  by 


5a 

0.012 

0 

0 

5e 

0 

0 

0 

6r 

0 

0 

0 

Rcycy 

= 

0 

0.008 

-0.014 

R<ycz 

0 

0.008 

0.014 

PI 

11.2 

0.168 

0.293 

P2 

11.2 

0.167 

0.293 

dp'c 

dq'c 

dr'c 


(3.57) 


3.4  Linear  Controllers 


In  thrust-borne  flight,  LQ  control  techniques  are  developed  and  then  extended  to 
include  the  effects  of  uncertainty  in  the  plant  and  measurements.  The  effects  of  wind 
are  also  considered. 


3.4.1  YAK-54 

In  thrust-borne  flight,  and  assuming  that  the  vehicle  is  in  hover  with  no  upward 
velocity  component  u  1 ,  four  velocity  controllers  are  used  to  control  the  YAK-54.  The 
controllers  are: 

•  U-velocity  controller  that  adjusts  vertical  velocity  with  the  throttle.  This  con¬ 
troller  corresponds  to  the  axial  subsystem. 

•  Vertical  roll  rate  controller  which  uses  the  ailerons  to  trace  either  a  commanded 
roll  rate  or  a  commanded  orientation  (</>„)  about  the  body-axis  x  direction.  The 
vertical  roll  rate  controller  corresponds  to  the  roll  subsystem. 
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•  W- velocity  controller  which  uses  the  elevator  to  adjust  the  velocity  in  the  body- 
axis  z  direction.  This  controller  corresponds  to  the  longitudinal  coordinate 
system. 


•  V-velocity  controller  uses  the  rudder  to  adjust  the  velocity  in  the  body-axis  y 
direction.  And  the  V-velocity  controller  corresponds  to  the  lateral  subsystem. 

Looking  first  at  the  U-velocity  controller,  the  subsystem  (3.48)  becomes 

u  1  I"  -0.038  ol  M  I"  25.8 

+  5t  (3.58) 

h  \  [  1  0  J  [  h  \  [  0 

when  numerical  values  are  determined  about  the  linearization  point.  Observing  that 
h  =  u,  an  LQR  controller  is  developed  that  also  incorporates  an  integral  term,  thus 
ensuring  zero-steady  state  error  properties  [6],  as  the  height  can  be  thought  of  as  the 
integral  of  the  u-velocity  component.  For  the  LI- velocity  controller,  weights  were  cho¬ 
sen  to  heavily  penalize  throttle  changes  and  deviations  in  u-velocity,  while  placing  less 
emphasis  on  aircraft  height.  The  weight  matrices  used  to  solve  the  Riccati  Equation 
(2.8)  and  that  penalize  state  errors  and  control  displacements  are 

10  0 

Q  =  ,  R  =  100,  (3.59) 

0  1 

which  lead  to  a  gain  of 

Kax  =  [  0.327  0.1  ]•  (3.60) 

The  throttle  control  input  can  be  thought  of  as 


5T~  °-327(“  U— *)  0-1/(W  u  command )  dt . 


(3.61) 


In  addition,  a  saturation  block  is  used  to  prevent  the  throttle  setting  from  dropping 
too  low  while  in  thrust-borne  flight.  The  saturation  block  is  set  at  50%,  and  is 
necessary  to  ensure  sufficient  airflow  and  dynamic  pressure  over  the  control  surfaces 
due  to  prop-wash.  A  block  diagram  of  the  throttle  controller  is  shown  in  Figure  3.2. 


35 


Figure  3.2:  Block  diagram  of  the  vertical  velocity  throttle  controller. 


The  roll  subsystem  (Equation  3.49),  controlled  via  the  roll  rate  controller  has 
weight  matrices  of 

r  1  0 
l  0  10 

which  give  a  gain  matrix  of 


Q  = 


,  R  =  1, 


(3.62) 


Kr  =  \  -1.0424  -3.6123 


(3.63) 


when  solving  the  Riccati  Equation  that  is  used  to  calculate  the  aileron  input.  The 
weights  for  adjusting  the  aileron  penalize  deviations  in  vertical  roll  angle,  the 
most.  The  aileron  input  is  then  given  by 


5a  =  1.0424p  +  3.6123(</>„  -  command)-  (3.64) 


The  longitudinal  subsystem  (3.50)  and  the  lateral  subsystem  (3.51)  both  make  use 
of  the  hybrid  LQR/integral  approach  used  in  the  development  of  the  vertical  velocity 
controller.  Weights  were  chosen  to  more  heavily  penalize  deviations  of  both  w  and 
9V,  while  also  penalizing  large  deflections  of  the  elevator.  For  the  longitudinal  system, 
the  weight  matrices  are 


Q 


10  0  0  0 
0  10  0 
0  0  100  0 
0  0  0  1 


R  =  10, 


(3.65) 
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and  the  gain  matrix  is 

Klong  =  [  0.578  -0.446  -3.776  0.316  ]•  (3.66) 

The  elevator  control  input  is  then 

5e  =  —0.578 (w  -  wcommand )  +  0.446g  +  3.776 9V  -  0.316  J (w  -  wcommand)dt.  (3.67) 

Similarly  for  the  lateral  subsystem  and  the  V-velocity  controller,  though  the  chosen 
weights  penalize  deviations  in  v,  i/)v,  and  Sr: 

10  0  0  0 
0  10  0 

Q=  ,  R  =  100  (3.68) 

0  0  1000  0 

0  0  0  1 

Kiat  =  [  -0.478  -0.447  -3.782  -0.1  ]  (3.69) 

Sr  =  0.478(u  -  V comrnond)  +  0.447r  +  3.782^  +  0.1  J (v  -  Vcommandjdt.  (3.70) 

For  thrust-borne  flight,  the  YAK-54  control  structure  is  shown  in  Figure  3.3. 

For  control  of  the  YAK-54  in  wing-borne  level  flight,  proportional  integral  (PI) 
controllers  are  used  to  control  the  rudder,  aileron,  elevator  and  throttle.  In  maintain¬ 
ing  level  flight,  the  rudder  is  used  to  eliminate  side  slip  of  the  aircraft.  Side  slip  is 
given  by 

(3  =  sin_1(n/K),  (3.71) 

where  Va  is  the  overall  velocity  vector  of  the  aircraft.  With  all  values  measured  in 
degrees,  the  rudder  control  is 

Sr  =  0.5  (3  +  -0.05  J  (3dt.  (3.72) 

The  aileron  is  controlled  by  a  wing-levelcr  that  tracks  a  commanded  bank  angle, 
<p.  For  wings- level  flight,  the  controller  tracks  a  zero  reference  value,  but  can  also  be 
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YAK-54  Dynamics 


Figure  3.3:  Block  diagram  of  the  control  structure  of  the  YAK-54  in  thrust-borne 
flight. 


used  to  track  bank  angles  for  turning  flight.  The  aileron  control  signal  is 

A  3(0  0 command )  Y  0.75  j  ( 0  0 command) dt .  (3.73) 

For  small  disturbances  from  trimmed  flight,  the  elevator  is  used  to  control  the 
measured  altitude  of  the  YAK-54  by  observing  the  altitude  and  tracking  a  reference 
altitude.  Again  using  a  PI  control  structure,  the  altitude  controller  gives  an  elevator 
control  of 

6e  =  0'5(/l  “  h~and)  ~  0-045  /(^  -  h command, ) dt •  (3.74) 

Finally,  the  throttle  is  used  to  control  the  airspeed  Va  of  the  aircraft  for  small 
disturbances  and  changes  from  trimmed  level  flight.  The  magnitude  of  the  airspeed 
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is 


Va=  {u2  +  v2  +  w2Y1/2\  (3.75) 

which  is  used  to  find  the  throttle  control  signal 

5t  =  "°-5(K  “  co~nd)  ~  °-°5  /(K  “  Va’  co~na ')dL  (3’76) 

3-4-2  Aerovel  Flexrotor 

The  development  of  the  Flcxrotor’s  controllers  for  thrust-borne  flight  closely  follows 
the  approach  used  for  the  YAK-54;  however,  different  weights  are  used  in  solving  the 
Riccati  Equation.  The  controllers  for  each  subsystem  are  of  the  form  u  =  —Kx: 


The  axial  subsystem  has  weights 


100 

0 

10 

0 

Q  = 

0 

0.01 

,  R  = 

0 

10 

(3.77) 


which  admit  a  gain  matrix  of 

K  = 

iyax 


0.93  -0.01 

-2.78  0.03 


(3.78) 


The  roll  subsystem  has  weights 

Q  = 

which  admit  a  gain  matrix  of 


10  0 
0  100 


R=  10, 


(3.79) 


K-roll 


1.65  3.16 


(3.80) 


The  longitudinal  subsystem  has  weights 

1000  0  0  0 


Q  = 


0  10  0 
0  0  100  0 
0  0  0  10 


R  =  1000, 


(3.81) 
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which  admit  a  gain  matrix  of 

Kiong  =  0.27  1.08  1.97  -0.03  •  (3.82) 

•  The  lateral  subsystem  has  weights 

1000  0  0  0 

0  10  0 

Q=  ,  R  =  100,  (3.83) 

0  0  100  0 

0  0  0  10 

which  admit  a  gain  matrix  of 

Kiat  =  4.52  4.03  20.8  0.316  .  (3.84) 

3.5  Open  Loop  Controllers 

In  order  to  develop  open  loop  controllers  for  transition  between  wing-borne  flight  and 
thrust-borne  flight,  a  few  details  must  to  be  considered.  First,  the  aircraft  design 
plays  a  large  role  in  choosing  a  desired  trajectory.  The  YAK-54  has  a  much  higher 
thrust-to-weight  ration  than  the  Flexrotor  so  it  can  accelerate  faster  and  to  higher 
speeds  in  the  vertical  direction,  allowing  a  greater  range  of  maneuvers.  The  Flexrotor, 
with  its  focus  on  long  range  and  endurance  is  less  agile,  so  specific  care  needs  to  be 
taken  to  develop  transition  maneuvers  suitable  for  the  aircraft.  Second,  the  cost  of 
transitions  needs  to  be  considered  so  that  the  maneuvers  can  be  improved  upon  by 
a  reinforcement  or  apprenticeship  learning  algorithm.  Possible  cost  functions  include 
minimizing  time  to  transition,  fuel  or  battery  power  consumed,  or  altitude  lost  during 
the  transition.  Finally,  some  hard  constraints  might  exist,  such  as  on  altitude  if 
operating  in  a  confined  area  or  a  minimum  altitude  for  the  Flexrotor’s  thrust-borne 
launch  /  recovery. 
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3.5.1  YAK-54 

With  the  YAK-54’s  high  thrust-to- weight  ratio,  many  possibilities  exist  for  transitions 
to  level  flight.  These  possibilities  include  climbing  in  an  accelerating  arc  to  sustained 
wing-borne  flight  that  would  be  accomplished  with  the  throttle  and  a  slow  elevator 
ramp,  while  the  ailerons  and  rudder  are  used  to  keep  the  wings  level.  A  much  faster 
maneuver  could  be  a  simultaneous  power  and  pitch  rate  reduction  to  induce  departure 
from  thrust-borne  flight,  then  use  the  elevator  to  recover  into  wing-borne  flight. 

For  transition  to  hover  from  wings-level  flight,  a  full  throttle  climb  combined  with 
a  slow  elevator  ramp  to  put  the  YAK-54  into  an  accelerating  vertical  climb  could 
be  used.  The  aircraft  would  then  reduce  u  velocity  until  stationary  hover  is  reached; 
however,  this  maneuver  would  be  an  inefficient  maneuver  given  its  power  and  altitude 
consumption.  A  faster  transition  could  be  made  by  rapid  pitch-up  past  the  stall 
regime,  using  the  aircraft’s  power  to  avoid  departure  from  controlled  flight  and  reach 
a  hover  configuration.  This  approach  would  require  much  less  altitude,  time,  and 
power  consumption. 

3.5.2  Aerovel  Flexrotor 

At  a  disadvantage  power  wise  when  compared  to  the  YAK-54,  multiple  choices  are 
nevertheless  present  for  transition  to  wing-borne  flight  for  the  Flexrotor.  The  first, 
a  hammerhead  transition  [19]  involves  a  slow  climb  in  the  vertical  direction  until 
sufficient  altitude  is  gained.  Then,  using  the  rudder  and  the  rotor  cyclic  in  the  y- 
body  axis  direction,  a  fast  yaw  would  be  commanded,  resulting  in  the  Flexrotor  in  a 
dive  with  its  nose  to  the  ground.  A  simple  pull-up  maneuver  with  the  elevator  would 
conclude  the  transition  to  level  flight.  Alternately,  a  twist-and-slew  transition  [19] 
would  use  the  rotor  cyclic  to  translate  the  aircraft  in  the  y-body  axis  direction.  As 
the  aircraft  gains  speed,  a  twist  performed  with  ailerons,  rudder,  and  cyclic  would 
rotate  the  Flexrotor  into  a  wings-level  configuration,  at  which  point  the  wing-borne 
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control  laws  would  dictate  control  inputs. 

The  Flexrotor’s  transition  to  hovering  flight  from  wings-level  flight  could  be  ac¬ 
complished  in  much  the  same  manner  as  the  YAK-54.  By  using  the  elevator,  as  well  as 
the  rotor  cyclic  in  the  z-body  axis  direction,  a  quick  pitch  up  followed  by  an  increase 
in  throttle  hover  could  be  reached. 

Ultimately,  for  both  aircraft,  it  is  desirable  to  have  a  human  pilot  demonstrate 
the  transition  maneuvers,  while  recording  his/her  commanded  control  inputs.  These 
inputs  should  then  be  used  in  simulation  with  the  apprenticeship-learning  algorithm 
to  determine  a  near-optimal  transition  for  the  aircraft. 

3.6  General  Control  Framework 

While  control  laws  are  specific  to  a  vehicle,  the  ideas  and  approach  used  in  this  thesis 
combine  to  form  an  appropriate  general  framework  for  control  of  UAVs  capable  of 
sustained  thrust-borne  flight.  The  proposed  framework  can  be  separated  into  three 
categories. 

3.6.1  Wing-Borne  Flight 

Control  of  the  aircraft  in  the  traditional  flight  modes  of  wing-borne  flight  should  be 
executed  according  to  the  well-documented  strategies  in  [30]  and  [25],  and  that  are 
used  in  UAVs  currently  in  operation.  Feedback  laws  such  as  pitch  attitude  hold, 
altitude  hold,  bank  angle  hold,  and  heading  hold  can  be  used  here. 

3.6.2  Thrust- Borne  Flight 

For  control  of  the  agile  UAV  in  thrust-borne  flight,  the  approach  taken  by  this  thesis 
is  recommended.  By  linearizing  the  aircraft  dynamics  about  a  hover  trim  point  and 
subsequently  decoupling  the  system,  control  laws  for  controlling  u,  v,  w,  and  (j)v  can 
be  generated  with  LQ  techniques,  and  if  required,  control  gains  can  subsequently  be 
tweaked  to  obtain  desired  controller  performance  characteristics. 
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3. 6. 3  Transitions 

Either  using  a  software  simulation,  a  hardware  simulation  in  the  loop  (HiL),  or  the 
UAV  with  a  human  pilot,  successful  open  loop  transitions  to  both  thrust-borne  and 
wing-borne  flight  should  be  captured.  While  these  open  loop  control  sequences  will 
work  to  navigate  between  the  two  flight  regimes,  by  applying  the  MDP  formulation 
to  the  system  and  operating  on  the  transitions  with  the  apprenticeship  learning  algo¬ 
rithm,  the  control  sequences’  optimality  can  be  improved. 
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Chapter  4 

SIMULATION  RESULTS 

The  motivating  problem  of  controlling  an  agile  autonomous  aircraft  capable  of 
sustained  thrust-borne  flight  is  demonstrated  here  in  simulation  using  the  various 
controllers  of  the  preceding  chapter.  Simulations  for  both  the  YAK-54  and  the  Aerovel 
Flexrotor  are  presented. 

The  chapter  is  composed  of  several  sections,  each  focusing  on  demonstrating  dif¬ 
ferent  controllers.  In  the  first  section,  the  thrust-borne  flight  controllers  for  both 
the  YAK-54  and  the  Flexrotor  will  demonstrate  the  performance  of  each  controller. 
Section  4.2  presents  simulations  for  the  wing-borne  flight  controllers  of  the  YAK-54. 
Next,  in  Section  4.3,  transition  controllers  are  simulated  for  the  YAK-54. 

All  simulations  were  done  in  MATLAB  with  ode45  on  an  Intel  Pentium  equipped 
computer  in  the  Autonomous  Flight  Systems  Lab  at  the  University  of  Washington. 

4.1  Thrust-Borne  Flight  Controllers 

The  thrust-borne  flight  controllers  using  the  LQ  approach  and  controllers  of  Section 
3.4  were  used  to  simulate  the  dynamics  of  the  agile  aircraft  systems. 

4-1.1  YAK-54 

The  simulations  for  the  YAK-54  were  initialized  from  a  trimmed  position  in  thrust- 
borne  flight  at  a  location  of  [0  0  200]  in  the  NED  frame. 

Figure  4.1  shows  the  throttle  response  to  a  commanded  2  meter  per  second  step 
input  in  u  velocity.  The  throttle  does  saturate  to  max  throttle  levels,  and  then 
returns  to  a  setting  slightly  higher  than  the  trim  setting.  The  saturation  does  not 
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Figure  4.1:  Nonlinear  U-velocity  controller  response  to  a  2  m/s  step  input  in  u  velocity 
for  the  YAK-54  demonstrating  a  fast  response  with  a  5%  overshoot. 


cause  concern,  as  it  is  for  a  short  duration  of  less  than  half  a  second.  Additionally, 
increasing  vertical  speed  is  a  “maximum  performance”  type  of  maneuver,  and  while 
the  YAK-54  has  a  sufficiently  high  thrust-to-weight  ratio,  accelerating  in  the  vertical 
direction  requires  significant  thrust.  The  response  in  u  velocity  is  nicely  shaped,  with 
a  5%  overshoot  and  fast  rise  and  settling  times. 

Figure  4.2  depicts  the  response  of  the  vertical  roll  rate  controller  for  the  YAK-54. 
At  five  seconds,  a  30  degree  step  input  in  commanded  (j)v  occurred.  As  with  the  throt¬ 
tle  step  response,  the  aileron  fully  saturates  for  a  short  period  of  time  to  its  maximum 
deflection  of  -35  degrees  before  settling  back  down  to  the  trim  setting  necessary  to 
counteract  the  rotational  effects  of  the  motor  and  prop  wash.  The  response  in  vertical 
roll  angle  demonstrates  no  oscillations  as  30  degrees  is  reached. 

Figure  4.3  displays  the  elevator  response  of  a  —3  m/s  step  input  in  w  velocity  for 
the  W-velocity  controller.  Following  a  quick  deflection,  the  elevator  settles  to  a  value 
of  10  degrees,  necessary  for  sustained  —3  m/s  translation  in  the  body  2-axis.  The 
controller  must  overcome  the  inertial  and  wind  resistance  of  the  wing  before  settling 
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Figure  4.2:  Nonlinear  vertical  roll  rate  controller  response  to  a  30  degrees  step  input  in 
commanded  vertical  roll  angle  (j)v  for  the  YAK-54.  The  control  signal  is  characterized 
by  a  fast  response  with  no  measurable  oscillations. 


Figure  4.3:  Nonlinear  W-velocity  controller  response  to  a  —3  m/s  step  input  in  w 
velocity  for  the  YAK-54  with  a  10%  overshoot. 


to  the  commanded  value  with  a  10%  overshoot. 

Figure  4.4  shows  the  response  of  the  YAK-54’s  V-velocity  controller  to  a  com¬ 
manded  step  input  of  3  m/s  in  v  velocity  at  5  seconds.  The  control  signal  dictates  a 
fast  rudder  doublet  before  reaching  a  trimmed  value  of  11  degrees  of  deflection.  The 
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Figure  4.4:  Nonlinear  V-velocity  controller  response  to  a  3  m/s  step  input  in  v  velocity 
for  the  YAK-54  with  a  21%  overshoot. 


doublet  is  necessary  to  initiate  the  tracking  of  the  reference  velocity.  The  response  in 
v  velocity  has  a  21%  overshoot  and  the  slowest  settling  time  of  the  four  controllers 
for  the  YAK-54. 

While  each  of  the  four  controllers  can  track  its  commanded  reference  signal,  all 
four  must  also  be  able  to  track  multiple  commanded  values  simultaneously.  Figure  4.5 
shows  the  input  traces  for  one  such  simulation  and  Figure  4.6  shows  the  corresponding 
control  traces  for  the  same  simulation.  As  before,  the  YAK-54  was  initialized  at 
[0  0  200]  in  the  NED  frame,  and  at  five  seconds,  step  inputs  were  commanded.  The 
multiple  step  inputs  were  -2  m/s  in  v  velocity,  2  m/s  in  w  velocity,  and  15  degrees  in 
vertical  roll  angle,  (j)v.  The  vehicle  does  successfully  track  the  commanded  inputs,  and 
its  position  moves  to  the  northwest,  ending  at  -25  meters  east  and  13  meters  north 
at  the  end  of  the  15  second  simulation.  The  YAK-54  also  loses  3.5  meters  of  altitude 
in  this  simulation.  The  U-velocity  controller  continues  to  track  the  commanded  value 
of  0  m/s,  so  the  YAK-54  gradually  drops  in  altitude  as  its  overall  velocity  vector, 
Va  —  [u  v  w]T ,  points  below  the  horizon. 

The  control  traces  of  Figure  4.6  depict  the  necessary  control  surface  deflections 
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Figure  4.5:  State  traces  for  the  YAK-54  in  thrust-borne  flight  showing  aircraft  re¬ 
sponse  to  three  commanded  step  inputs  occurring  at  five  seconds.  The  step  inputs 
are  -2  m/s  in  v  velocity,  2  m/s  in  w  velocity,  and  15  degrees  in  vertical  roll  angle. 
The  simulation  was  initialized  at  [0  0  200]  in  the  NED  frame. 


to  track  the  commanded  step  inputs,  as  well  as  settling  to  new  trim  values.  After 
the  step  inputs,  the  elevator  reaches  a  new  trim  state  of  -15  degrees,  after  a  positive 
deflection  from  its  previous  state  of  -7  degrees.  The  aileron  affects  the  vertical  roll 
rate  necessary  to  track  the  input,  and  then  returns  to  its  trim  value  of  just  over  -5 
degrees  of  deflection.  The  small  difference  in  the  trim  values  of  the  aileron  setting  is 
due  to  the  v  and  w  components  of  aircraft  velocity  affecting  the  tendency  to  rotate 
the  YAK-54  about  its  x-body  axis.  The  rudder,  initially  trimmed  at  zero  degrees, 
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reaches  a  new  trim  state  of  -9  degrees.  Finally,  the  throttle  setting  only  drops  slightly 
as  the  U-velocity  controller  continues  to  track  a  zero  u  velocity. 
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Figure  4.6:  Control  traces  for  the  YAK-54  in  thrust-borne  flight  necessary  to  track 
three  commanded  step  inputs  occurring  at  five  seconds  and  corresponding  to  the  state 
traces  in  Figure  4.5.  The  step  inputs  are  -2  m/s  in  v  velocity,  2  m/s  in  w  velocity, 
and  15  degrees  in  vertical  roll  angle.  The  simulation  was  initialized  at  [0  0  200]  in  the 
NED  frame. 


4-1.2  Aerovel  Flexrotor 

The  simulations  for  the  Aerovel  Flexrotor  and  its  four  thrust-borne  flight  controllers 
developed  in  Section  3.4  used  the  linear  system,  as  the  non-linear  dynamics  were  not 
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accessible  in  the  Matlab/Simulink  environment.  While  the  controllers  were  developed 
for  the  diagonalized  and  decoupled  linear  system,  the  full  linear  system  was  used  in 
simulation.  Additionally,  rather  than  the  diagonalized  control  inputs  dp'c,  dq'c ,  and 
dr'c ,  the  mixing  matrix  was  used  to  map  to  the  physical  control  most  closely  associated 
with  each  controller. 

Figure  4.7  demonstrates  the  Flexrotor’s  response  to  a  2  m/s  step  input  in  vertical 
velocity.  The  U-velocity  controller  does  not  exhibit  any  overshoots,  while  exhibiting 
a  fast  response.  As  the  Flexrotor  has  a  smaller  thrust-to-weight  ratio  compared  to 
the  YAK-54,  accelerating  in  the  vertical  direction  is  a  difficult  maneuver,  and  the 
response  shown  is  too  fast  for  the  physical  system. 


Figure  4.7:  Linear  U-velocity  controller  response  for  the  Aerovel  Flexrotor  to  a  2  m/s 
step  input  in  commanded  u  velocity.  The  response  does  not  exhibit  any  overshoot. 


Figure  4.8  shows  the  tip  thruster  response  to  a  30  degree  step  input  in  commanded 
vertical  roll  angle.  The  vertical  roll  rate  controller  commands  slightly  over  18  watts 
of  power  for  the  thrusters  to  track  the  step  input,  out  of  an  available  75  watts. 
The  response  in  vertical  roll  angle  does  not  have  any  overshoot  and  settles  to  the 
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commanded  value  in  7.5  seconds. 


Figure  4.8:  Linear  vertical  roll  rate  controller  response  for  the  Flexrotor  to  a  30 
degrees  step  input  in  commanded  vertical  roll  rate 


Figure  4.9:  Linear  W-velocity  controller  response  for  the  Flexrotor  to  a  —3  m/s  step 
input  in  w  velocity. 


Figure  4.9  demonstrates  the  response  of  the  hybrid  rotor /propeller  of  the  Flexrotor 
to  a  commanded  step  input  in  w  velocity.  The  W-velocity  controller  commands  small 
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changes  for  the  cyclic  in  the  z-body  axis  direction  to  effect  the  necessary  change  in 
velocity.  To  effect  the  desired  response,  the  controller  must  first  reach  positive  w 
velocity  before  it  is  able  to  settle  to  the  commanded  velocity  at  18  seconds. 


Figure  4.10:  Linear  V-velocity  controller  response  for  the  Flexrotor  to  a  3  m/s  step 
input  in  v  velocity  with  a  21%  overshoot. 


Figure  4.10  depicts  a  much  faster  response  via  the  cyclic  in  the  y-body  axis  direc¬ 
tion  to  a  commanded  step  input  in  v  velocity.  The  response  of  the  V-velocity  controller 
has  a  30%  overshoot,  and  three  oscillations  before  settling  to  the  commanded  value 
of  3  m/s. 

4.2  Wing-Borne  Flight  Controllers 

The  response  of  the  wing-borne  level  flight  controllers  for  the  YAK-54  is  shown  in 
Figures  4.11  and  4.12.  This  full-state  nonlinear  simulation  was  initialized  at  [0  0  200] 
in  the  NED  frame,  and  depicts  the  aircraft  response  as  the  PI  controllers  reach  a 
trimmed  state. 

The  state  traces  show  small  oscillations  in  the  aircraft  velocities,  orientation,  and 
altitude.  Additionally,  the  YAK-54  tracks  slightly  the  west  of  north  in  the  simulation, 
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Figure  4.11:  State  traces  for  the  YAK-54  in  wing-borne  flight  showing  aircraft  re¬ 
sponse  as  the  wing-borne  flight  controllers  settle  the  aircraft  into  a  wings-level  flight 
trimmed  condition.  The  simulation  was  initialized  at  [0  0  200]  in  the  NED  frame. 


following  a  heading  angle  of  355.5  degrees.  This  heading  change  results  from  the 
increase  in  bank  angle  after  the  simulation  begins,  and  the  heading  angle  settles  to 
steady  state  as  the  bank  angle  returns  to  zero  degrees.  The  coupling  between  pitch 
angle  and  altitude  can  also  be  seen,  as  the  oscillations  are  quite  similar  as  the  signals 
settle  to  steady-state. 

The  control  traces  of  Figure  4.12  demonstrate  oscillations  that  are  small  in  mag¬ 
nitude  as  well.  The  elevator  deflection  varies  between  -6.38  and  -6.51  degrees  over 
50  seconds,  while  the  aileron  varies  between  0.05  and  -0.14  degrees.  The  changes  in 
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Figure  4.12:  Control  traces  for  the  YAK-54  in  wing-borne  flight  corresponding  to  the 
state  traces  in  Figure  4.11.  The  simulation  was  initialized  at  [0  0  200]  in  the  NED 
frame. 


rudder  and  throttle  are  negligible. 
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Chapter  5 

CONCLUSION 


5.1  Summary 

This  thesis  has  addressed  the  problem  of  autonomous  control  of  the  class  of  agile  air¬ 
craft  capable  of  performing  sustained  thrust-borne  flight  by  looking  specifically  at  two 
different  platforms,  the  University  of  Washington  YAK-54  and  the  Aerovel  Flexrotor. 
The  approach  taken  was  built  upon  new  and  existing  examinations  of  thrust-borne 
control.  First,  the  problem  of  system  dynamics  and  modeling  was  addressed.  Then, 
the  systems  were  linearized  and  decoupled  so  that  linear  controllers  could  be  devel¬ 
oped  for  thrust-borne  flight.  For  the  YAK-54,  level  flight  was  also  considered.  Finally, 
open  loop  control  methods  were  examined  for  both  aircraft.  Simulation  results  were 
shown  to  support  the  developed  controllers. 

These  results  present  a  new  approach  to  the  control  of  vehicles  capable  of  au¬ 
tonomous  hover,  in  addition  to  examining  a  radical  new  concept  in  the  Flexrotor. 
Prior  work  was  varied  and  many  different  approaches  and  techniques  discussed.  This 
thesis  aimed  to  examine  previous  work  and  validate  previous  techniques  while  propos¬ 
ing  a  general  framework  for  problems  of  this  nature. 

5.2  Future  Work 

The  work  in  this  thesis  is  limited  in  scope  as  no  tests  involving  hardware-in-the-loop 
or  actual  flight  test  were  performed.  While  the  work  presented  here  is  a  necessary 
step  towards  implementing  and  testing  these  controllers  on  the  YAK-54,  much  work 
remains  to  be  done.  For  example,  the  wing-borne  flight  controllers  need  more  devel¬ 
opment  and  a  guidance  and  navigation  scheme  for  level  flight  must  be  considered. 
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Additionally,  the  YAK-54  must  be  fully  instrumented  and  prepared  for  autonomous 
flight  (see  Figure  5.1),  with  all  controllers  finalized,  including  the  near-optimal  tran¬ 
sition  sequences  found  via  apprenticeship  learning. 


Figure  5.1:  The  University  of  Washington  YAK-54  agile  aircraft. 


In  order  to  progress  to  flight  test,  the  suggested  approach  is  to  develop  a  tether 
system  to  test  the  hover  controllers  prior  to  untethered  autonomous  flight.  Addition¬ 
ally,  a  fail-safe  switch  to  return  control  of  the  YAK-54  to  a  human  pilot  should  be 
considered.  Successful  flight  testing  of  the  YAK-54  using  apprenticeship  learning  se¬ 
quences  would  represent  the  first  instance  of  the  algorithm  being  used  for  autonomous 
aircraft  aerobatics,  though  the  apprenticeship  learning  approach  has  been  shown  to 
work  for  an  autonomous  helicopter. 

Another  direction  for  future  work  would  be  to  implement  a  target  tracking  ability 
that  would  require  transition  between  flight  regimes.  An  extension  could  also  be  the 
use  of  outer  loop  search  algorithms,  such  as  the  one  presented  in  [13]  and  [14].  Fur¬ 
ther  application  of  autonomous  hover  control  techniques  to  specific  problems  such  as 
search,  constricted  space  maneuvers,  or  object  avoidance  would  validate  the  approach 
discussed  in  this  thesis. 
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